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ABSTRACT 

This paper offers a look into the future of 
educational program auditing and presents a detailed discussion of 
the notion of a districtwide educational audit* To date, virtually 
all program audit activities have occurred as a result of specially 
funded State or federal education programs. Once the program ends, or 
changes its requirements, the audit is eliminated. The author argues 
that the only way in vhich educational program auditing can have a 
long-range impact on improved decisionmaking and accountability is to 
have the audit conducted at the school district level as a regular 
and continuing activity* (Author/DN) 
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EDUCATIONAL PROGRAM AUDITING: PERSPECTIVES ON THE FUTURE* 

Marvin C. Alkin 
• Center for The Study of Evaluation 

Educational program audits, as we have known them, exhibit three common 
characteristics* First, they have, almost without exception, been Federally 
mandated. Second, because of their association with a specifically funded 
project, theyare shortlived* Finally, their final reports must be directed 
towards meeting legal requirements. 

Educational audits were originally established as a part of ESEA Title 
VII (Bilingual Education) and Title VIII (Dropout Prevention), The regulatio 
governing the programs stated that schools or districts benefiting from the 
Federal funds "'...must provide for an independent educational accomplishment 
audit of the project to apprise school officials of the validity of their own 
evaluative processes and data (Title VII Manual p.ll)." In many of the dis- 
tricts, the typical pattern was for the evaluation to be conducted internally 
either with the project director performing the evaluation functions himself 
or, in the case of larger projects, with other district staff conducting the 
evaluation. The audit, as then conceptualized, performed a function of pro- 
viding external information to FederaMevel program monitors. 

Federal funding of education projects is generally of short duration (2- 
3 years). When funding is terminated, or a project reaches its conclusion, 
auditors no longer are utilized. Thus, audits are performed only during a 
very limited period of time. 

" — ^ 
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The audience for program audit reports is a restricted one. Ordinarily, 
evidence is presented in a final report for the expressed purpose of enabling 
project officers and USOE personnel to determine whether a project has com- 
plied with its legal obligations and to certify the objectivity of the project 
evaluation. Thus, the auditor must write a report which is satisfactory to 
the FedercO monitor and which complies with Federal requirements, formats, etc., 
while at the same time maintaining ttie good graces of the local project direc- 
tor who hired the auditor and will hopefully do so again in the future. 

Because it is imposed on the schools, is a temporary activity, and is re- 
ported to a limited audience, one might well conclude that the educational 
program audit has a minimum impact potential and holds little promise for wide- 
spread future use. 

This might be the case but for several factors. Even the most optimitic 
educators cannot consider, the credibility of schools to be flourishing. This 
lack of credibility and a concern for demonstrating that schools are doing an 
adequate job have led legislators, school boards, parents > and other concerned 
individuals down the thorn-strewn path of grasping at misleading information. 
This has been readily demonstrated by the attempts of legislators and school 
boards, in particular, to compare scores on standardized tests to affirm their 
contentions about the effectiveness of local schools. Usually, such simple- 
minded analyses— using tests that may not measure district goals and which do 
not consider differences in district input characteristics—provide misleading 
infomiation. The outcome of such analyses is more confusion rather than en- 
lightenment and the public still in a position of wanting to know ^*how our 
schools ar^ doing.'' 
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In short, the educational program audit iTiight remain merely a part of the 
complex Federal machinery were it not for the impact of a word which is heard 
in steadily increasing volume: ACCOUNTABILITY. Clearly, there is a growing 
need to present to the public, in a manner particularly suited to its concerns, 
verified evidence of the impact of schools upon children. The educational 
program audit promises to serve as a means to this end, 

* Many of us are aware of the functions performed by financial auditing 
firms in the examination of the books of major business firms in the United 
States and elsewhere. It has become common practice for every major company 
to employ a financial auditor to present a yearly report on the "financial 
health and well-being" of the company. These yearly reports by Peat, Marwick, 
Mitchell & Co.t Price Waterhouse & Co., and other giants of the accounting 
field are presented at yearly stockholder meetings and represent a guarantee 
by an external agency that "all is well" and that the statements and assertions 
of management can and have been externally validated. Beyond this, the finan- 
cial auditing report may go on to pinpoint deficiencies of the accounting sys- 
tem that might be remedied by the initiation of a new practice. Thus, one 
area of the yearly financial auditing report points to the adequacy of the 
accounting system for providing information for decision making. This system 
has worked well for industry and the business world—it has provided necessary 
assurances to the stockholders and boards of directors that the system is oper- 
ating properly. Financial auditing is the appropriate tool for the examination 
of industry, because the main function of industry (the major output) is profit . 
But the major function of a governmental institution, and in particular a public 
educational institution, is not profit making. Certainly one must be aware of 
the costs associated with various aspects of the enterprise, but the major output 
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of public educational Institutions is their impact upon children. Thus, while 
the publics of industrial firms seek assurances that the firm's statements 
about financial well-being are accurate, the publics of local school districts 
also must be regularly assured of the "educational well-being*' of these insti- 
tutions. 

To consider how an educational program audit might provide this assurance, 
let us proceed from a number of principles. First, an educational audit Is 
not an evaluation. Just as a financial audit is not a total accounting system, 
an educational audit cannot perform the total evaluation function. The institu- 
tions involved in an audit must still maintain their own educational accounting 
system— a system of management and evaluation. This system should provide in- 
formation and allow for management decisions and change. 

Second, certain modifications are necessary to adapt current auditing pro- 
cesses, which were developed for use in Federal projects, to a different set of 
constraints. For example, in contrast to present mandates, the future educa- 
tional audit would serve as a tool to be used in a voluntary manner. In addi- 
tion, it is suggested that the impetus for carrying out an audit would no longer 
originate at the Federal level, but would arise from the local district level. 
School district officials, anxious to lend credence to data about the review, 
reform, or results of various programs, could contract with an educational audit 
team to accomplish this mission. Another projected change in current practice 
would be a move away from the short-term audit. It is highly improbable that a 
single educational audit could encompass all programs of interest to a community. 
Therefore, an audit would likely become an iterative process , covering major 
areas of concern over a number of years. Moreover, to accommodate a different 
set of decision maker concerns, the nature of the audit functions themselves 
would have to be modified. 



Audit Functions 

Thus far, we have considered only the general characteristics vfhich might 
be associated with future audits. It has been stated that an audit cannot pro- 
vide a complete and comprehensive evaluation of every program within a district. 
What, then, can be done? What are the possible functions of a district-wide 
audit? 

The audit team could be responsible for the examination and verification 
of district-prepared evaluations . The auditors could consider evaluation re- 
ports on existing educational programs that have either been prepared within 
the district or external to the district and verify the appropriateness of the 
procedures employed in the evaluation, as well as verify the findings or point 
to deficiencies where they exist. Naturally, this could take place only to the 
extent that such documents are available. But despite the paucity of formal 
evaluation documents performed by local districts, there are many attempts by 
districts to gather data of an "evaluative*' nature. By reviewing internal eval- 
uation procedures and results, the auditors could provide assurances about the 
objectivity of this Information. 

The audit team could collects analyze^ and evaluate a sampling of programs 
within the district . It has been previously mentioned that it is not possible 
to evaluate each and every program of a school district. Programs of highest 
importance or interest probably would be identified by district personnel as 
areas of concern to be studied during the audit. It is expected that over the 
course of several years most areas would have been evaluated. The design of 
collection procedures and the selection of instruments would be major concerns 
of one auditor. The auditor would also perform all analyses and could prepare 
the report of the evaluation. The auditor should maintain the option of selec- 
ting additional programs for study In order to ensure the integrity of the audit, 
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The audit team couTld examine and critique the adequacy of the district's 
management and evaluation system . As ''outside" agents, the auditors are in an 
excellent position to recommend modifications of the system by which the dis- 
trict acquires and utilizes evaluation information. Thus, the third function 
of the future audit would be the examination of the procedures employed in the 
management and evaluation system of the school district and the consideration 
of the adequacy of that system for producing current and accurate evaluation 
information and for making program changes based upon that information. The 
auditors would ask, does the school district's administration have an adequate 
system for obtaining data on how the various educational programs are doing, 
and how might that system be improved? AUeration of existing systems would 
be designed to maximize constructive use of data obtained during the audit as 
well as information derived from other evaluative sources. 

These three new educational audit functions can be referred to as evalua- 
tion verification , evaluation sampling and management and evaluation system . 
One could st^ite that each of these functions helps to establish an accountabil- 
ity system, i.e., helps to determine the extent to which an educational insti- 
tution is fulfilling its responsibilities. This would be a hollow assertion in- 
deed, without further discussion of the meaning of accountability. 

The Audit and Accountability 

If a district program audit is to be examined in light of its potential 
usefulness in an accountability system* one must first resolve two critical 
issues: 

Who is accountable to whom? 
For what are they accountable? 
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My response to these questions is a proposal for goal , progra m and outcome 
accountability which has been set forth elsewhere (Alkin, 1972; Baker and 
Alkin, 1973). In these scheme, school boards and school district adminis- 
tration are held accountable tc the public for the proper selection of 
goals ("goal accountability"). Administrators and other professional staff 
are held accountable to the school boards for maintaining a program appro- 
priate for meeting stipulated objectives ("program accountability"). 
Teachers are held accountable to administrators and professional staff for 
the attainment by pupils of specified learning outcomes ("outcome account- 
ability'*). Diagram 1 depicts these three levels of accountability and the 
corresponding responsibilities of each group involved in the instructional 
process. 

The Stages of an Audit 

A district program audit may be conducted in a way which effectively 
analyzes each of the levels of accountability described above • In order to 
accomplish this, it is suggested that such an audit be implemented in three 
stages. Chart I shows each of these three stages; the operational activi- 
ties of the audit team at each stage are discussed below. 
Stage 1, Goal and Program Selection Audit 

The audit team determines where sets of operationally defined 
goal statements are available at the district level. The team then 
analyzes the extent to which these goals are translated to various 
operating units within the district and whether instructional sys- 
tems said to be in use in the district appear to have face validity 
in terms of their attaining the stipulated goals. These activities 
help provide information for the reporting of goal accountability 
and, to some extent, program accountability. 



stage 2. Program Implementation Audit 

The audit team determines whether educational pisans, procedures 
and systems stipulated in district guides, policy manuals and other 
documents have been implemented into school practice. This proce- 
dure is concerned with program accountability. 
Stage 3. Outcome Audit 

The audit team determines whether educational outputs of the 
system are in accord with intended educational goals and assesses 
whether student performance is at a satisfactory leval. These 
activities deal with outcome accountability. 

Application of the District Audit Model 

The district program audit model previously discussed was implemented 
recently in a California school district. The district superintendent was con- 
vinced of the importance of a yearly audit to verify the "well being'* of the 
' educational system to the board of education. My colleagues and I developed 
the audit model and devised specific procedures for its implementation. The 
essential content of the audit is summarized on Chart II. 

In order to keep the cost of the initial audit within reasonable limits, 
a two-year time span was designated for the completion of the three audit 
stages (goal and program, implementation, and outcome). During Year 1, two 
reports were written. Report I dealt exclusively with goal and program audit 
considerations and was primarily a review of district policy manuals, educa- 
tional plans, and other documents. District and school goal and program state- 
ments were examined for comprehensiveness, clarity, and internal and intra- 
document consistency. 
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Report II, just recently completed, emphasized implementation audit issues 
(although some goal and program and outcome auditing was also done). The imple- 
mentcition audit concerned Uself with whether educational plans reflecting dis- 
trict goals were implemented into school practice. The auditors met with the 
school superintendent and, to some extent, with all school baord members to as- 
certain which areas of interest should be the subject of this stage of the au- 
dit. Among several others, programs for the mentally gifted, special activities, 
and a reading support system were identified as topics of concern. 

Implementation information was acquired, in part, by interviews conducted 
with a representative sample of randomly selected school personnel. The inter- 
views focused on the key questions derived from district documents and agreed 
upon by district personnel and the auditors. Examples of these questions are 
as follows: 

1. What is the purpose of (this program)? 

2. Are all eligible students participating in (the special 
program)? If not, why not? 

The auditors were thus responsive to the concerns of the district, but also 
maintained the option of adding supplemental questions to assure the objectivity 
of the audit. By comparing the results of the interviews with the written des- 
criptions of the programs in question, discrepancies between implementation and 
documents could easily be identified. 

In addition to the interview, observation instruments were designed to 
verify the accuracy of the interview with regard to three of the programs under 
study. In each case, district personnel were notified only of the date and 
school in which an observation would take place, but were not informed of the 
program to be observed. 
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Report III, to be completed within the next few months, will highlight 
outcome audit information and will treat briefly issues in the remaining two 
stages of the audit plan. Several areas in which district evaluative data 
are available will undergo evaluation verification in v;hich the auditor ex- 
amities the data, s>pot checks its authenticity where possible, and confirm::; 
the validity of the findings. In addition, the adequacy of the district pro- 
cedures for collecting information for decision ttiaking wlill be examined with 
suggestions made by the auditors as to means of improving this evaluation in- 
formation system. Finally, several program areas will be selected jointly 
by the auditor and the district administration and school board for the con- 
duct of a sample evaluation by the auditor. 

The projected Annual Audit Report in subsequent years will devote approx- 
imately equal time to implementation and outcome audit stages with some small 
consideration each year to goa] and program auditing. Chart II depicts the 
overall District Audit Implementation Plan as described above. 

Clearly, there is a new mechanism for satisfying the demands for demon- 
strating accountability of total schoDl programs— THE DISTRICT PROGRAM AUDIT . 

References 

Alkin, M.C. Accountability defined. Evaluation Comment , 1972, 1-5. 

Baker^ E.L.* & Alkin, M.C. Formative evaluation of instructional development. 
AV Communication Review , 1973, 21(4), 389-418. 

Title VII Manual for Grantees and Applicants. U.S. Office of Education, 
Department of Health, Education, and Welfare, Washington, 1971. 



ERIC 



10 



Diagram 1 

THE THREE LEVELS OF ACCOUNTABILITY 




1. Outcome Accountability 



ADMINISTRATORS, PROFESSIONAL STAFF 
• responsible for maintaining program 
appropriate for mecMinj; objectives 
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2, Pro.uram Ac^ 



SCHOOL BOARDS 
responsiijle for propr.T selcciion of goals' 




3. Goal Accountability 



PUBLIC 

responsiljle for ruviow of school lioard policy and actions 
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